Computer systems exhibiting improved computer speed and transcription accuracy of automatic speech transcription (ast) based on a multiple speech-to-text engines and methods of use thereof

ABSTRACT

In some embodiments, an exemplary inventive system for improving computer speed and accuracy of automatic speech transcription includes at least components of: a computer processor configured to perform: generating a recognition model specification for a plurality of distinct speech-to-text transcription engines; where each distinct speech-to-text transcription engine corresponds to a respective distinct speech recognition model; receiving at least one audio recording representing a speech of a person; segmenting the audio recording into a plurality of audio segments; determining a respective distinct speech-to-text transcription engine to transcribe a respective audio segment; receiving, from the respective transcription engine, a hypothesis for the respective audio segment; accepting the hypothesis to remove a need to submit the respective audio segment to another distinct speech-to-text transcription engine, resulting in the improved computer speed and the accuracy of automatic speech transcription and generating a transcript of the audio recording from respective accepted hypotheses for the plurality of audio segments.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.17/153,575, filed Jan. 20, 2021, which is a continuation of U.S. patentapplication Ser. No. 16/208,291, filed Dec. 3, 2018, now U.S. Pat. No.10,930,287, which is a continuation of U.S. patent application Ser. No.15/933,040, filed May 30, 2018, now U.S. Pat. No. 10,147,428. Theaforementioned applications are hereby incorporated herein by referencein their entirety.

FIELD OF THE INVENTION

The present invention relates generally to computer systems exhibitingimproved computer speed and transcription accuracy of automatic speechtranscription based on a multiple speech-to-text engines and methods ofuse thereof.

BACKGROUND

Typically, an automatic (non-human) speech recognition/transcription(AST) converts speech to text via a single speech recognition model.Typically, such single speech recognition model only transcribes asingle language using a single acoustic model and is based on trade-offsin accuracy between general and specific language patterns.

SUMMARY OF THE INVENTION

In some embodiments, the present invention provides for an exemplaryinventive computer-implemented method for improving computer speed andaccuracy of automatic speech transcription that includes at least thesteps of: generating, by at least one processor, at least one speechrecognition model specification for a plurality of distinctspeech-to-text transcription engines; where each distinct speech-to-texttranscription engine corresponds to a respective distinct speechrecognition model; where, for each distinct speech-to-text transcriptionengine, the at least one speech recognition model specification at leastidentifies: i) a respective value for at least one pre-transcriptionevaluation parameter, and ii) a respective value for at least onepost-transcription evaluation parameter; receiving, by the at least oneprocessor, at least one audio recording representing at least one speechof at least one person; segmenting, by the at least one processor, theat least one audio recording into a plurality of audio segments; wherein each audio segment corresponds to a respective single phrase of arespective single person that has been bounded by points of silence inthe at least one audio recording; determining, by the at least oneprocessor, based on the respective value of the at least onepre-transcription evaluation parameter of the respective distinct speechrecognition model in the at least one speech recognition modelspecification, a respective distinct speech-to-text transcription enginefrom the plurality of distinct speech-to-text transcription engines tobe utilized to transcribe a respective audio segment of the plurality ofaudio segments; submitting, by the at least one processor, therespective audio segment to the respective distinct speech-to-texttranscription engine; receiving, by the at least one processor, from therespective distinct speech-to-text transcription engine, at least onehypothesis for the respective audio segment; accepting, by the at leastone processor, the at least one hypothesis for the respective audiosegment based on the respective value of the at least onepost-transcription evaluation parameter of the respective distinctspeech recognition model in the at least one speech recognition modelspecification to obtain a respective accepted hypothesis for therespective audio segment of the plurality of audio segments of the atleast one audio recording; where the accepting of the at least onehypothesis for each respective audio segment as the respective acceptedhypothesis for the respective audio segment removes a need to submit therespective audio segment to another distinct speech-to-texttranscription engine from the plurality of distinct speech-to-texttranscription engines resulting in the improved computer speed and theaccuracy of automatic speech transcription; generating, by the at leastone processor, at least one transcript of the at least one audiorecording from respective accepted hypotheses for the plurality of audiosegments; and outputting, by the at least one processor, the at leastone transcript of the at least one audio recording.

In some embodiments, the generating of the at least one speechrecognition model specification for the plurality of distinctspeech-to-text transcription engines includes: receiving, by the atleast one processor, at least one training audio recording and at leastone truth transcript of the at least one training audio recording;segmenting, by the at least one processor, the at least one trainingaudio recording into a plurality of training audio segments and the atleast one truth transcript into a plurality of corresponding truthtraining segment transcripts; applying, by the at least one processor,at least one pre-transcription audio classifier to each training audiosegment of the plurality of training audio segments to generate firstmetadata classifying each training audio segment based at least on: i)language, ii) audio quality, and iii) accent; applying, by the at leastone processor, at least one text classifier to each corresponding truthtraining segment transcript of the plurality of corresponding truthtraining segment transcripts to generate second metadata classifyingeach corresponding truth training segment transcript based at least onat least one content category; combining, by the at least one processor,the plurality of training audio segments, the plurality of correspondingtruth training segment transcripts, the first metadata, and the secondmetadata to form at least one benchmark set; testing, by the at leastone processor, each distinct speech-to-text transcription engine of theplurality of distinct speech-to-text transcription engines based on theat least one benchmark set to form a plurality of model result sets;where each model result set corresponds to the respective distinctspeech-to-text transcription engine; where each model result setincludes: i) the at least one benchmark set, ii) at least onemodel-specific training hypothesis for each training audio segment, iii)at least one confidence value associated with the at least onemodel-specific training hypothesis, and iv) at least one word error rate(WER) associated with the at least one model-specific traininghypothesis; determining, by the at least one processor, a respective setof transcription decisions for each distinct speech-to-texttranscription engine of the plurality of distinct speech-to-texttranscription engines, where the respective set of transcriptiondecisions defines, for each distinct speech-to-text transcriptionengine, the value of the at least one pre-transcription evaluationparameter and the value of the at least one post-transcriptionevaluation parameter; and combining, by the at least one processor, eachrespective set of transcription decisions for each distinctspeech-to-text transcription engine of the plurality of distinctspeech-to-text transcription engines into the at least one speechrecognition model specification for the plurality of distinctspeech-to-text transcription engines.

In some embodiments, the at least one pre-transcription evaluationparameter is selected from the group of: i) the language, ii) the audioquality, and iii) the accent.

In some embodiments, the respective set of transcription decisionsincludes at least one of: i) a pre-transcription importance ranking of aplurality of pre-transcription evaluation parameters, and ii) apost-transcription importance ranking of a plurality ofpost-transcription evaluation parameters.

In some embodiments, the at least one post-transcription evaluationparameter is a confidence threshold.

In some embodiments, each segment of the plurality of audio segmentslasts between 5 and 15 seconds.

In some embodiments, the at least one audio recording is real-timestreamed audio of the at least one speech of the at least one person.

In some embodiments, the at least one audio recording includes at leasttwo speeches of at least two people; where the generating the at leastone transcript of the at least one audio recording further includes:generating a first transcript of a first speech of a first person, andgenerating a second transcript of a second speech of a second person;and where the outputting the at least one transcript of the at least oneaudio recording further includes: outputting the first transcript of thefirst speech of the first person, and outputting the second transcriptof the second speech of the second person.

In some embodiments, the at least two speeches are in distinctlanguages.

In some embodiments, the respective distinct speech recognition model isselected from the group consisting of: i) a phoneme-based acousticGaussian mixture model, ii) a phoneme-based acoustic hidden Markovmodel, iii) a phoneme-based acoustic neural net model trained fromforced phonetic alignments, iv) a phoneme-based acoustic neural netmodel trained without forced phonetic alignments, v) a character-basedacoustic neural net model, vi) any of i-v models coupled with an n-gramlanguage model, and vii) any of i-v models coupled with a generative,neural net language model.

In some embodiments, the present invention provides for an exemplaryinventive system for improving computer speed and accuracy of automaticspeech transcription that includes at least components of: at least onespecialized computer, including: a non-transient computer memory,storing particular computer executable program code; and at least onecomputer processor which, when executing the particular program code, isconfigured to perform at least the following operations: generating atleast one speech recognition model specification for a plurality ofdistinct speech-to-text transcription engines; where each distinctspeech-to-text transcription engine corresponds to a respective distinctspeech recognition model; where, for each distinct speech-to-texttranscription engine, the at least one speech recognition modelspecification at least identifies: i) a respective value for at leastone pre-transcription evaluation parameter, and ii) a respective valuefor at least one post-transcription evaluation parameter; receiving atleast one audio recording representing at least one speech of at leastone person; segmenting the at least one audio recording into a pluralityof audio segments; where in each audio segment corresponds to arespective single phrase of a respective single person that has beenbounded by points of silence in the at least one audio recording;determining, based on the respective value of the at least onepre-transcription evaluation parameter of the respective distinct speechrecognition model in the at least one speech recognition modelspecification, a respective distinct speech-to-text transcription enginefrom the plurality of distinct speech-to-text transcription engines tobe utilized to transcribe a respective audio segment of the plurality ofaudio segments; submitting the respective audio segment to therespective distinct speech-to-text transcription engine; receiving, fromthe respective distinct speech-to-text transcription engine, at leastone hypothesis for the respective audio segment; accepting the at leastone hypothesis for the respective audio segment based on the respectivevalue of the at least one post-transcription evaluation parameter of therespective distinct speech recognition model in the at least one speechrecognition model specification to obtain a respective acceptedhypothesis for the respective audio segment of the plurality of audiosegments of the at least one audio recording; where the accepting of theat least one hypothesis for each respective audio segment as therespective accepted hypothesis for the respective audio segment removesa need to submit the respective audio segment to another distinctspeech-to-text transcription engine from the plurality of distinctspeech-to-text transcription engines resulting in the improved computerspeed and the accuracy of automatic speech transcription; generating atleast one transcript of the at least one audio recording from respectiveaccepted hypotheses for the plurality of audio segments; and outputtingthe at least one transcript of the at least one audio recording.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention, briefly summarized above anddiscussed in greater detail below, can be understood by reference to theillustrative embodiments of the invention depicted in the appendeddrawings. It is to be noted, however, that the appended drawingsillustrate only typical embodiments of this invention and are thereforenot to be considered limiting of its scope, for the invention may admitto other equally effective embodiments.

FIGS. 1-5B are representative of some exemplary aspects of the presentinvention in accordance with at least some principles of at least someembodiments of the present invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Among those benefits and improvements that have been disclosed, otherobjects and advantages of this invention can become apparent from thefollowing description taken in conjunction with the accompanyingfigures. Detailed embodiments of the present invention are disclosedherein; however, it is to be understood that the disclosed embodimentsare merely illustrative of the invention that may be embodied in variousforms. In addition, each of the examples given in connection with thevarious embodiments of the present invention is intended to beillustrative, and not restrictive.

Throughout the specification, the following terms take the meaningsexplicitly associated herein, unless the context clearly dictatesotherwise. The phrases “in one embodiment” and “in some embodiments” asused herein do not necessarily refer to the same embodiment(s), thoughit may. Furthermore, the phrases “in another embodiment” and “in someother embodiments” as used herein do not necessarily refer to adifferent embodiment, although it may. Thus, as described below, variousembodiments of the invention may be readily combined, without departingfrom the scope or spirit of the invention.

In addition, as used herein, the term “based on” is not exclusive andallows for being based on additional factors not described, unless thecontext clearly dictates otherwise. In addition, throughout thespecification, the meaning of “a,” “an,” and “the” include pluralreferences. The meaning of “in” includes “in” and “on.”

It is understood that at least one aspect/functionality of variousembodiments described herein can be performed in real-time and/ordynamically. As used herein, the term “real-time” is directed to anevent/action that can occur instantaneously or almost instantaneously intime when another event/action has occurred (e.g., less than 1 seconddifference between sequential events/actions).

As used herein, the term “dynamic(ly)” means that events and/or actionscan be triggered and/or occur without any human intervention.

As used herein, the term “computer engine” identifies at least onesoftware component and/or a combination of at least one softwarecomponent and at least one hardware component which aredesigned/programmed/configured to manage/control other software and/orhardware components (such as the libraries, software development kits(SDKs), objects, etc.).

In some embodiments, events and/or actions in accordance with thepresent invention can be in real-time and/or based on a predeterminedperiodicity of at least one of: nanosecond, several nanoseconds,millisecond, several milliseconds, second, several seconds, minute,several minutes, hourly, etc.

In some embodiments, the inventive adaptive self-trained computerengines with associated devices may be configured to operate in thedistributed network environment, communicating over a suitable datacommunication network (e.g., the Internet, etc.) and utilizing at leastone suitable data communication protocol (e.g., IPX/SPX, X.25, AX.25,AppleTalk™, TCP/IP (e.g., HTTP), etc.). Of note, the embodimentsdescribed herein may, of course, be implemented using any appropriatehardware and/or computing software languages. In this regard, those ofordinary skill in the art are well versed in the type of computerhardware that may be used, the type of computer programming techniquesthat may be used (e.g., object oriented programming), and the type ofcomputer programming languages that may be used (e.g., C++, Basic, AJAX,Javascript). The aforementioned examples are, of course, illustrativeand not restrictive.

Examples of hardware elements may include processors, microprocessors,circuits, circuit elements (e.g., transistors, resistors, capacitors,inductors, and so forth), integrated circuits, application specificintegrated circuits (ASIC), programmable logic devices (PLD), digitalsignal processors (DSP), field programmable gate array (FPGA), logicgates, registers, semiconductor device, chips, microchips, chip sets,and so forth. In some embodiments, the one or more processors may beimplemented as a Complex Instruction Set Computer (CISC) or ReducedInstruction Set Computer (RISC) processors; x86 instruction setcompatible processors, multi-core, or any other microprocessor orcentral processing unit (CPU). In various implementations, the one ormore processors may be dual-core processor(s), dual-core mobileprocessor(s), and so forth.

Examples of software may include software components, programs,applications, computer programs, application programs, system programs,machine programs, operating system software, middleware, firmware,software modules, routines, subroutines, functions, methods, procedures,software interfaces, application program interfaces (API), instructionsets, computing code, computer code, code segments, computer codesegments, words, values, symbols, or any combination thereof.Determining whether an embodiment is implemented using hardware elementsand/or software elements may vary in accordance with any number offactors, such as desired computational rate, power levels, heattolerances, processing cycle budget, input data rates, output datarates, memory resources, data bus speeds and other design or performanceconstraints.

One or more aspects of at least one embodiment may be implemented byrepresentative instructions stored on a machine-readable medium whichrepresents various logic within the processor, which when read by amachine causes the machine to fabricate logic to perform the techniquesdescribed herein. Such representations, known as “IP cores” may bestored on a tangible, machine readable medium and supplied to variouscustomers or manufacturing facilities to load into the fabricationmachines that actually make the logic or processor.

In one example implementation, a multi-processor system may include aplurality of processor chips each of which includes at least one I/Ocomponent. Each of the processor chips may also include a voltageregulation circuit configured to regulate a voltage of one or more ofthe processor chips. In some embodiments, the voltage regulation circuitof each of the processor chips may provide one or more voltage domainsof the respective processor chip. In some embodiments, themulti-processor system may further include one or more additionalelectronic components, e.g., inductors, as part of the package. In someembodiments, the multi-processor system may include at least 10,000processor chips and may be packaged into a total volume of no more than8 m.sup.3. In some embodiments, a density of the processor chips may begreater than 1,000 chips per cubic meter. In some embodiments, a latencyof the multi-processor system, having more than 1,000 processor chips,may be less than 200 nanoseconds (ns).

In one example implementation, a multi-processor system may include aninter-processor interconnect (IPI) and a plurality of processor chips.The processor chips are configured to communicate data to one anotherthrough the IPI. Each of the processor chips may include one or morecores and one or more level 1 (L1) caches. Each of the L1 caches may beassociated with a respective core through a respective core-cachebandwidth. Each of the processor chips may also include at least onememory controller and one or more local memory devices. Each of thelocal memory devices may be associated with the at least one memorycontroller through a respective local memory bandwidth. Each of theprocessor chips may further include an on-chip interconnect (OCI) thatis associated with the one or more cores and the at least one memorycontroller of that processor chip. The OCI is also associated with theIPI of the multi-processor system. The association between the OCI andthe plurality of cores of that processor chip is through a bandwidththat is greater than 50% of an aggregate core bandwidth, which isapproximately the sum of each core-cache bandwidth of that processorchip. The association between the OCI and the at least one memorycontroller of that processor chip is through a bandwidth that is greaterthan 50% of an aggregate memory bandwidth, which is approximately thesum of each local memory bandwidth of that processor chip. Theassociation between the OCI and the IPI of the multi-processor system isthrough an injection bandwidth. In some embodiment, the injectionbandwidth is greater than 50% of the aggregate core bandwidth of thatprocessor chip. In some embodiment, the injection bandwidth is greaterthan 50% of a sum of the aggregate core bandwidth and the aggregatememory bandwidth of that processor chip.

In some embodiments, the exemplary inventive multi-engine AST system maybe programmed/configured to acquire and/or process audible speechrecordings from one or more users which may be, but is not limited to,at least 100 (e.g., but not limited to, 100-999), at least 1,000 (e.g.,but not limited to, 1,000-9,999), at least 10,000 (e.g., but not limitedto, 10,000-99,999), at least 100,000 (e.g., but not limited to,100,000-999,999), at least 1,000,000 (e.g., but not limited to,1,000,000-9,999,999), at least 10,000,000 (e.g., but not limited to,10,000,000-99,999,999), at least 100,000,000 (e.g., but not limited to,100,000,000-999,999,999), at least 1,000,000,000 (e.g., but not limitedto, 1,000,000,000-10,000,000,000).

In some embodiments, the exemplary inventive multi-engine AST system maybe configured to generate datasets of training data from samples ofhuman-generated speech and associated text to self-train the exemplaryinventive multi-engine AST system to generate, from data representativeof electronically acquired human speech (e.g., real-time processing),electronic messages and/or actions such as, but not limited to, messagesand/or actions related to execution of financial transactions (e.g.,trading of financial assets (e.g., stock, currency, bitcoin, physicaland/or virtual commodities, etc.).

In some embodiments, the exemplary inventive multi-engine AST system maybe configured to be trained during an iterative learning process when auser voices a set of representative number of words related to financialtrades. In some embodiments, the exemplary inventive multi-engine ASTsystem may be configured to record and store the user's speech into aleast one audio file or database entry.

In some embodiments, the exemplary inventive multi-engine AST system maybe configured to utilize audio files representative of variouscombinations of, for example, financial trades within the domain ofnumbers and typical trade keywords, as training data. In someembodiments, the exemplary inventive multi-engine AST system may beconfigured/programmed to utilize the audio files for training bydetermining at least one speaker identity signature of a particular userbased, at least in part, on frequency spectrum of voice samples recodedin the audio files.

In some embodiments, as referenced herein, the term “hypothesis” isdirected to a particular transcription output from a singlespeech-to-text engine utilizing a respective single speech recognitionmodel, representing a transcript of a particular audio input/recording.In some embodiments, each hypothesis may be outputted with correspondingmetadata such as, but not limited to, accuracy confidence metric(s) thatis/are representative of how confident the particular engine is that theparticular hypothesis is the accurate transcript of the correspondingaudio input/recording.

In some embodiments, the accuracy confidence metric(s) may be computedbased one of the following or any other similarly suitable method: asentence-level confidence calculated via a difference between the totalcosts of the best and the second best hypotheses, a sentence-levelconfidence calculated by a phoneme-weighted average of individual wordconfidences derived from a word confusion matrix of hypotheses, asentence-level confidence calculated by a character-weighted average ofindividual word confidences derived from a word confusion matrix ofhypotheses, sentence-level confidence calculated based on a product ofn-gram probabilities from a particular language model, a differencebetween the Levenshtein differences between characters in the besthypothesis and the highest probable phrase from a particular n-gramlanguage model.

In some embodiments, the accuracy confidence metric returned along withthe hypothesis of the transcript from the engine may be a statisticalcorrelation coefficient of 0.5 or greater when correlated against theaccuracy of the transcript. In some embodiments, the accuracy confidencemetric returned along with the hypothesis of the transcript from theengine may be a statistical correlation coefficient of 0.6 or greaterwhen correlated against the accuracy of the transcript. In someembodiments, the accuracy confidence metric returned along with thehypothesis of the transcript from the engine may be a statisticalcorrelation coefficient of 0.7 or greater when correlated against theaccuracy of the transcript. In some embodiments, the accuracy confidencemetric returned along with the hypothesis of the transcript from theengine may be a statistical correlation coefficient of 0.8 or greaterwhen correlated against the accuracy of the transcript. In someembodiments, the accuracy confidence metric returned along with thehypothesis of the transcript from the engine may be a statisticalcorrelation coefficient of 0.9 or greater when correlated against theaccuracy of the transcript.

In some embodiments, as referenced herein, the term “benchmark set” isdirected to an inventive compilation/library/data structure that atleast includes a set of digitized audio recordings/files, a set ofrespective transcripts (hypotheses), and any respective metadata (e.g.,accuracy confidence metric(s)).

In some embodiments, the exemplary inventive multi-engine AST system isconfigured to receive, during a training phase, a set of audiorecordings and accompanying transcribed text (i.e., benchmark set) thatmay be tagged according to, but not limited to, one or more contentproperties in a hierarchy of importance (e.g., language, audio quality,accent, topic of conversation, etc.) and be subject to user's preferenceand/or self-determination for pre- and post-transcription processingsettings (e.g., the pre-transcription importance ranking: language,audio quality, and accent; the post-transcription importance ranking: atopic of conversation). In some embodiments, the exemplary inventivemulti-engine AST system may be configured so that an array oftranscription engines are utilized to transcribe the audio files in theinventive benchmark set and return one or more hypotheses and respectiveconfidence metric(s) per particular hypothesis.

In some embodiments, for example, the array of speech-to-texttranscription engines could encompass engines that are based on one ormore of the following distinct speech recognition models: aphoneme-based acoustic Gaussian mixture model, a phoneme-based acoustichidden Markov model, a phoneme-based acoustic neural net model trainedfrom forced phonetic alignments, a phoneme-based acoustic neural netmodel trained without forced phonetic alignments, a character-basedacoustic neural net model, any of the above mentioned models coupledwith an n-gram language model, any of the above mentioned models coupledwith a generative, neural net language model.

In some embodiments, the exemplary inventive multi-engine AST system isconfigured to determine the accuracy of hypotheses returned by eachengine by comparing to the benchmark set.

In some embodiments, the exemplary inventive multi-engine AST system isconfigured to address a computer-based technological problem associatedwith a desire to utilize multiple speech recognition models to improveefficiency and/or accuracy of the AST because distinct speechrecognition models may return distinct metrics, and a typical ASTcomputer processor, receiving hypotheses from these distinct speechrecognition models, does not know which model would return the besthypothesis. Further, yet another computer-based technological problemexists because evaluating hypotheses from these distinct speechrecognition models by the typical AST computer processor would becomputationally expensive since such evaluation would typically entailthat every audio input piece would require to be transcribed multipletimes in parallel.

In some embodiments, to address at least the above identifiedtechnological problems, the exemplary inventive multi-engine AST systemis configured to select a base speech-to-text engine programmed based onat least one speech recognition model from the array of speech-to-textengines, thus allowing to obtain a transcript with the fewest errorsthan any of distinct speech recognition models without submitting everypiece of audio recording to multiple transcription engines. In someembodiments, the exemplary inventive multi-engine AST system isconfigured to compare the accuracy of the base engine to the remainingengines within each content property category in the pre-transcriptioncategory to determine according to each parameter if an alternativeengine hypothesis exceeds the accuracy of the base engine. In someembodiments, the exemplary inventive multi-engine AST system isconfigured which improves computer processor(s) of a typical AST systemby allowing multi-language conversations to be transcribed by a singlecomputer processor leveraging multiple transcription models/engines. Insome embodiments, the exemplary inventive multi-engine AST system isconfigured which improves computer processor(s) of a typical AST systemby allowing multi-language conversations to be contemporaneously (at thesame relative time period) transcribed by leveraging multipletranscription models/engines. In some embodiments, the exemplaryinventive multi-engine AST system is configured which improves computerprocessor(s) of a typical AST system by allowing multi-languageconversations to be simultaneously (at the same relative time moment)transcribed by leveraging multiple transcription models/engines.

In some embodiments, the exemplary inventive multi-engine AST system isconfigured to compare the accuracy of the base engine to the remainingengines across all confidence levels for the post-transcription contentproperties to determine at what confidence level for each category thebase engine's hypothesis should be rejected in favor of an alternativehypothesis from an alternative engine having the higher accuracy. Insome embodiments, the exemplary inventive multi-engine AST system isconfigured to utilize the accuracy confidence metrics from both the baseengine and alternative engine as decision boundaries.

In some embodiments, the exemplary inventive multi-engine AST system isconfigured to utilize a content decision tree generated based, at leastin part, on thresholds for pre-transcription content types and/oraccuracy confidence-thresholds for post-transcription content types.

In some embodiments, the exemplary inventive multi-engine AST system isconfigured to split, during a decoding phase/stage, an input audiorecording/file into segments of speech where each segment encompasses asingle phrase (two or more words) of a single speaker. In someembodiments, words of the phrase form a (e.g., a component of a clause).In some embodiments, a particular conceptual unit does not form acomplete sentence.

In some embodiments, the exemplary inventive multi-engine AST system isconfigured to utilize a pre-transcription audio classifier to determine,for each segment of the inputted speech, at least a language or adialect spoken, an audio quality, and an accent of the speaker.

In some embodiments, the exemplary inventive multi-engine AST system isconfigured to submit each audio file for each segment to the basetranscription engine that would meet pre-determined parameters of thelanguage, audio quality, and accent based on the inventive contentdecision tree generated during the training phase.

In some embodiments, the exemplary inventive multi-engine AST system isconfigured to aggregate hypothesis/transcripts from all segments and themultiple engines into a single final transcript.

FIG. 1 shows a diagram of an exemplary innovative method of creating theexemplary inventive benchmark set in according to at least someembodiments of the present invention. For example, the exemplaryinventive multi-engine AST system is configured to receive an inputaudio recording with accompanied “truth” text transcript. For example,the exemplary inventive multi-engine AST system is configured to segmentthe input audio recording into a plurality of audio files havingportions of the input audio recording, where each portion (a singleaudio file) has duration of X seconds and corresponds to asingle-speaker spoken phrase (101) of the accompanied transcript boundedby points of silence. In some embodiments, X may vary from 5 to 15seconds. In some embodiments, X may vary from 1 to 60 seconds. In someembodiments, X may vary from 3 to 30 seconds. In some embodiments, X mayvary from 5 to 20 seconds. In some embodiments, X may vary from 5 to 10seconds.

Referring to FIG. 1 , the exemplary inventive multi-engine AST system isconfigured to process the segmented audio files (102) through anexemplary audio classifier (104) for generating associated metadata(tagging) describing audio parameter(s)/characteristic(s) of each audiofile, such as, but not limited to, language, audio quality, and accent(the first metadata). In some embodiments, the exemplary audioclassifier (104) may be any suitable machine-learned model that wouldtake input audio and outputs a label (e.g., “English” as language label,“Clear” as audio quality label, “New York” as accent label). In someembodiments, the set of labels output by the machine-learned model maybe assigned to numeric values within a defined range (e.g. 0.1 to 5.0),where higher or lower values are determined to be either superior orinferior, such that superior labels can be compared to inferior labelsin a numeric way when appropriate (e.g. audio quality). In someembodiments, the exemplary audio classifier may take in input audiofiles and tags them based on a hierarchy of audio characteristics, withthe first characteristic, for example, being the most important in thespeech recognition model selection. In some embodiments, the hierarchyof audio characteristics may be defined as language of the audio (themost important), then the audio quality, then the accent of the speaker(the least important). In some embodiments, the language of the audioand the accent of the speaker may be determined via an exemplaryclassification model that has been trained from a set of audio withlabels of the language and/or accent they contain, where the features ofthe audio may be modeled using, for example, twenty mel-frequencycepstrum coefficients. In some embodiments, the exemplary classificationmodel may be one of a machine-learned neural-network-type model, supportvector machine, naïve Bayes model, or any model with an accuracy of 90%or higher on identifying the correct label. In some embodiments,depending on the source of the audio, the accuracy may be of 95% orhigher. In some embodiments, depending on the source of the audio, theaccuracy may be of 99% or higher. In some embodiments, depending on thesource of the audio, the accuracy may be of 80% or higher. In someembodiments, depending on the source of the audio, the accuracy may beof 85% or higher. In some embodiments, the language and accentclassifiers may be two independently trained classifiers, or a singleclassifier that outputs a composite label. In some embodiments, theaudio quality may be scored, for example, without limitation, via thesum of the sample rate, signal to noise ratio, and product of the numberof peaks within the human voice spectrum and the degree of clipping,less the mean amplitude. For example, each value in the audio qualitycalculation may be normalized by a weight. For example the audio qualitycan be defined as:

${{AQ}\left( {r,s,p,d,m} \right)} = {\frac{r}{w_{r}} + \frac{s}{w_{s}} + \frac{p*d}{w_{p}} - \frac{m}{w_{m}}}$

where “r” is the sample rate, “s” is the signal to noise ratio, “p” isthe number of peaks within the human voice spectrum, “d” is the degreeof clipping (measured as the number of samples within 5% of the maximumvalue over the total number of samples), and “n” is the mean amplitude.For example, the weights (e.g., w_(r), w_(s), w_(p), w_(m)) may bedetermined as values that maximize a correlation of the audio quality toaccuracy within the benchmark set and simultaneously maintain, forexample, a value between 0.1 and 5.0.

In some embodiments, the exemplary inventive multi-engine AST system isalso configured to process, for example, in parallel, the corresponding“truth” text transcripts (103) of the segmented audio files (102)through an exemplary text classifier (105) to also generate additionaltext-related metadata (tagging) such as, but not limited to, the contentcategory and other similarly suitable features of each text (the secondmetadata). In some embodiments, the exemplary text classifier (105) maybe any suitable machine-learned model that would take input text andoutputs a label (e.g., “fashion”, “health”, “food”, “price”, etc.)

In some embodiments, the exemplary text classifier (105) may be aclassification model trained from a set of text with labels of thecontent types, where the features of the text may be, for examplewithout limitation, the one-to-three word phrase that include each setof text. In some embodiments, the classification model may be one of amachine-learned neural-network type model, support vector machine, naïveBayes model, or any model with an accuracy of 90% or higher onidentifying the correct label. In some embodiments, depending on thesource of the text, the accuracy may be of 95% or higher. In someembodiments, depending on the source of the text, the accuracy may be of99% or higher. In some embodiments, depending on the source of the text,the accuracy may be of 80% or higher. In some embodiments, depending onthe source of the text, the accuracy may be of 85% or higher. Forexample, the content types may describe the types of phrases within alanguage that are represented in the text, such as general conversationEnglish, financial-domain focused English, medical terminology, personalidentification information, and so on. In some embodiments, theexemplary inventive multi-engine AST system may be configured to utilizetranscripts containing alternate phonetic possibilities to improve thedetermination of the content type by the exemplary text classifier(105).

In some embodiments, the exemplary inventive multi-engine AST system isconfigured to combine the first and the second metadata, as, for examplewithout limitations, as JSON-formatted metadata with corresponding audiofiles and transcripts to form the benchmark set (106). In someembodiment, the benchmark set (106) may be stored in the form of

Referring to FIG. 2 , during the training stage, the exemplary inventivemulti-engine AST system is configured to benchmark variousspeech-to-text engines, each corresponding to one or more of speechrecognition models. For example, the exemplary inventive multi-engineAST system is configured to utilize the benchmark set (201), which isthe same benchmark set (106) from FIG. 1 , as a model result set (202)associated with a particular speech recognition model. For example, theexemplary inventive multi-engine AST system is configured to submit theaudio files (203) to the particular speech recognition model (204),which in turn returns at least one hypothesis of the transcript (205)and accuracy confidence metric(s) (206). For example, the exemplaryinventive multi-engine AST system is configured to combine thehypothesis (205) and the accuracy confidence metric(s) (206) with the“truth” transcript (207) from the model result set (202) and submit themto a word error rate (WER) calculator (208), the specialized softwareroutine. In some embodiments, the WER calculator (208) is configured todetermine the level of error in each hypothesis compared to thecorresponding “truth” transcript (i.e., the transcript of the audio fileas determined by a human listener), and combines the WER, confidencemetric(s), and the hypothesis into a data object (209) that theexemplary inventive multi-engine AST system is configured to merge withthe original metadata in the model result set (202) to form anbenchmarked model result set. For example, the exemplary inventivemulti-engine AST system is configured to repeat the same process witheach speech recognition model of each speech-to-text engine.

In some embodiments, each particular speech recognition model can be anysuitable machine-learned model that takes input audio files and returnsat least one hypothesis of the transcript. In some embodiments, theconfidence metric (e.g., confidence score) may be computed by theindividual speech recognition model and may use any suitable computationmethod so long as the confidence metric (e.g., confidence score) scorewould be between 0 and 1. In some embodiments, the confidence score maybe based on the log-likelihood difference between the first and secondmost likely similar but distinct hypotheses that exist for a given inputaudio. For example, a valid confidence score correlates with theaccuracy with a correlation of at least 0.6. For example, the validconfidence score correlates with the accuracy with a correlation of atleast 0.5. For example, the valid confidence score correlates with theaccuracy with a correlation of at least 0.7. For example, the validconfidence score correlates with the accuracy with a correlation of atleast 0.8. For example, the valid confidence score correlates with theaccuracy with a correlation of at least 0.9.

In some embodiments, regarding the WER calculator (208) is configured tocompare each hypothesis to the “truth” transcript and to score thenumber errors in the hypothesis based on the sum of substituted,deleted, and inserted words in the hypothesis as compared to the “truth”transcript. In some embodiments, the sum of errors may be normalized bythe total number of words in the “truth” transcript to obtain the WER:

${WER} = \frac{S + D + I}{N}$

where “S” is the number of substitutions, “D” is the number ofdeletions, “I” is the number of insertions and “N” is the number ofwords in the “truth” transcript. For example, if the “truth” transcriptof “testing one two three” is compared to a hypothesis of “testing oneone to three”, the sum of substituted and inserted words over the totalwords in the initial transcript is 2 divided by 4, or a 50% WER.

Referring to FIG. 3 , as part of the training stage, the exemplaryinventive multi-engine AST system is configured to determine boundaryconditions of the leaves of a decision tree for evaluating thetranscription from two speech recognition models. For example, theexemplary inventive multi-engine AST system is configured to obtain themodel result sets (from FIG. 2 ) for Model A (301) and Model B (302) andthen pass them thought its exemplary analysis engine (303) to create thedecision tree boundaries. For example, the exemplary analysis engine(303) is configured to complete its analysis in two parts: makingpre-transcription decision(s) based on the audio signal metric(s) forselection of a base model (304) and post-transcription decision(s) onthe content category/type of the text for determination of hypothesisacceptance or rejection based on the confidence metric(s) (305). In someembodiments, for each set of transcription decisions, the WER from themodel result set is used to determine the best model (a model exceedsanother model at speech recognition (transcription) when WER of itshypothesis is lower than WER of the other model's hypothesis).

Further referring to FIG. 3 , the analysis engine 303 thus comparesModel A's model result set (301) and Model B's model result set (302) todetermine the content categories/types (such as, but not limited to,general English, financial-domain language, financial trade, address,date, etc.) that each model exceeds at transcribing. Further referringto FIG. 3 , at the first phase, the analysis engine (303) is configuredto score each speech recognition model on the pre-transcriptionevaluation parameter(s)/characteristic(s) of the audio signal (e.g.,audio content language, audio content accent, and audio quality). Foreach characteristic, the result sets of speech recognition models areanalyzed to determine values of the labels at which a particular speechrecognition model exceeds the other model(s) in performance as measuredby the lowest WER. In some embodiments, for label-basedcharacteristics/metrics (e.g., language, accent), the modelspecification may simply include a qualification that the correspondingmodel is capable of transcribing that content by observing a hypothesisthat has a sufficient accuracy (e.g., <50% WER). In some embodiments,for threshold-based characteristics/metrics, a value for eachcharacteristic (e.g., 0.1 to 5 for audio quality; 0 to 1 for theconfidence) is saved to be used to choose the base model for aparticular audio segment.

Further referring to FIG. 3 , during the second phase, the analysisengine (303) is configured to score speech recognition model on thepost-transcription evaluation parameter(s)/characteristic(s) of theirresulting text: for example, the content type of the transcript. In someembodiments, a confidence threshold (CT) may range from 0.1 to 5 foreach speech recognition model in order to determine which transcript touse for a given audio segment. For example, the CT may be the thresholdat which a transcript from a particular speech recognition model with aconfidence lower than the threshold would be rejected. For example, insome embodiments, the exemplary inventive multi-engine AST system may beconfigured to determine the WERs of the base model and the alternativemodel for different CTs per model for subsets of the benchmark setscorresponding to each content type and audio quality in the availablelabels. For example, in some embodiments, CTs may be tuned to minimizethe overall WER of each subset of the benchmark data, by accepting allhypotheses from the base model whose confidences are larger than thebase model CT and accepting all hypotheses from the alternate modelwhose confidences exceed the alternate model CT. For example, if the WERdefined for CTs is lower than each individual model's WER, the CT'sspecifications are accepted as valid. In some embodiments, utilizing theCTs for both models, the entire benchmark set may be re-evaluated todetermine the overall WER if the CTs had been used to determine the besthypothesis. If the WER utilizing the CTs is lower than each individualmodel's WER, the CT specifications are accepted as valid.

In some embodiments, the resulting decision boundaries that theexemplary inventive multi-engine AST system is configured to utilize,based, at least in part, on the audio content and transcription content,to select a particular speech recognition model over another model arestored as model specification data structures/files (306) that can beindividually maintained per model or be composite specification for twoor more models. For example, a model could have a boundary value foraudio quality of 2.5 and a confidence boundary value of 0.8, indicatingthat audio must have an audio quality greater than 2.5 to be transcribedby the specified model, and the resulting hypothesis must have aconfidence value higher than 0.8 in order to be accepted.

Referring to FIG. 4 , the exemplary inventive multi-engine AST system isconfigured to utilize at least one speech recognition modelspecification covering one or more distinct speech recognition models(e.g., 306) to automatically transcribe a new audio recording by makingdecisions about speech recognition model selection and hypothesisacceptance. For example, the exemplary inventive multi-engine AST systemmay include an exemplary composite transcription engine (404) that wouldreceive, obtain and/or store each model specification file (401) thatwould contains model specification for each speech recognition model(e.g., the model A (402) and the model B (403) in this example). Asdetailed herein, the model specification file 401 that would include thepre-transcription and post-transcription classification specificationsfor both model A (402) and model B (403) such as both audio-levelclassification boundaries (such as audio quality) that are used todetermine the base model for the first pass of the transcription andtext content-level classification boundaries that are used to determinewhether a hypothesis should be accepted or rejected. In someembodiments, similarly to FIG. 2 , the exemplary inventive multi-engineAST system is configured to begin the transcription process of the newaudio recording (405) by segmenting the audio recording into audiofiles, where each audio file corresponds to a single phrase of a singlespeaker. In some embodiments, the exemplary inventive multi-engine ASTsystem is configured to feed each segmented audio file into theexemplary composite transcription engine (404). Then, the exemplarycomposite transcription engine (404) is configured to proceed inaccordance with exemplary decision process (406) to evaluate eachindividual segment.

Referring to FIG. 4 , in some embodiments, the composite transcriptionengine (404) is configured to determine the best model for a given audiosegment in a two-phase process. For example, in the first phase, thecomposite transcription engine (404) is configured to select a modelthat is more likely to produce the most accurate hypothesis through onlythe analysis of the audio and without the need to transcribe from allavailable engines, saving computational time as opposed to analyzing allpossible hypotheses from all engines. For example, the first phase ofthe analysis that the composite transcription engine (404) may performis based at least in part on calculating a metric “Q” for each speechrecognition model:

Q(v,x)=q(r,s,p,d,m)*l(w)*a(w)

v=(w,r,s,p,d,m)

x=(m _(b) ,m _(l) ,m _(a) ,m _(c))

where “w” is a vectorized representation of the features of the audiofile; “r”, “s”, “p”, “d”, and “m” are the sample rate, signal to noise,frequency peaks in the human voice range, degree of clipping, and meanamplitude, respectively; “x” is the model specification for a givenmodel; “m_(b)” is the audio quality boundary of the model, “mi” is theset of acceptable languages that the model can transcribe at WER of lessthan 50%, “m_(a)” a is the set of acceptable accents that the model cantranscribe at WER of less than 50%; “m_(c)” is the set ofcontent-dependent confidence boundaries; “q”, “l” and “a” are functionsthat evaluate the audio quality, language, and accent, respectively, ofthe audio file as shown below based on the input audio vector and/or themodel specification parameters:

${q\left( {r,s,p,d,m,m_{b}} \right)} = \left\{ \begin{matrix}{1,} & {{{AQ}\left( {r,s,p,d,m} \right)} \geq m_{b}} \\{0,} & {{{AQ}\left( {r,s,p,d,m} \right)} < m_{b}}\end{matrix} \right.$${l\left( {w,m_{l}} \right)} = \left\{ \begin{matrix}{1,} & {{L(w)} \in m_{l}} \\{0,} & {otherwise}\end{matrix} \right.$${a\left( {w,m_{a}} \right)} = \left\{ \begin{matrix}{1,} & {{A(w)} \in m_{a}} \\{0,} & {otherwise}\end{matrix} \right.$

In the above example, “AQ” is the audio quality function presentedpreviously, “L(w)” is the language classifier that takes the input audiofeature vector and returns a label of the language, and “A(w)” is theaccent classifier that takes the input audio feature vector and returnsa label of the accent.

For example, the available speech recognition models for transcriptionbased on “Q” can be found as the set “T” given “v” (audio parameters):

T(c)={x∈M|Q(v,x)=1}

where “x” is the set of model specifications as above and “M” is the setof model specifications for all available models. In some embodiments,if the set “T” contains multiple models, the one with the lowest audioquality boundary “m_(b)” would be chosen.

In some embodiments, once a particular speech recognition model has beenchosen by the composite transcription engine (404) and the model returnsat least one hypothesis and the confidence level for the transcriptionof the respective audio segment, the composite transcription engine(404) is configured to evaluate a second metric “R” to determine if asecond pass of transcription should occur. In some embodiments, thissecond phase metric R may be defined as follows:

u = C(t) ${R\left( {c,m_{c},u} \right)} = \left\{ \begin{matrix}{1,{c \geq {m_{c}(u)}}} \\{0,{c < {m_{c}(u)}}}\end{matrix} \right.$

where “u is the content type determined by passing the text “t” into thetext classifier “C”, “c” is the confidence metric returned from theinitial transcription, “m_(c)” is the confidence boundary dependent onthe content type “u” as determined in the model specification. If “R”equals “1”, then the hypothesis is accepted. If “R” equals “0”, thenanother model is chosen for the second phase of transcription. Forexample, the composite transcription engine (404) is configured todetermine one or more speech recognition models out of the availablemodels “V” for second pass of transcription, based, at least in part, onthe content type of the text:

V(u)={x∈T|x[m _(c)(u)]>0.01},

where “x” is the model specification, “T” is the set of speechrecognition (transcription) models previously found without the modelfirst used for transcription, and “x[m_(c)(u)]” is the confidenceboundary of the specific content type detected for each modelspecification “x”. If the confidence is greater than the minimum value,such model would be considered a viable model by the compositetranscription engine (404). If multiple models are contained within set“V”, then the composite transcription engine (404) is configured toselect the one with the highest confidence “m_(c)(u)” as best for thesecond phase transcription.

In some embodiments, once the second pass of transcription has occurredwith the at least one second model and at least second hypothesis andconfidence metric have been returned by the respective model, thecomposite transcription engine (404) may be configured to utilize thesame metric “R” to evaluate the second hypothesis. If “R” equals “0” forthe second hypothesis, then, the composite transcription engine (404)may be configured to accept the first model (i.e., previously rejectedat the first pass with “R” equals “0”). If “R” equals “1”, then thecomposite transcription engine (404) is configured to accept the secondhypothesis.

For example, the exemplary decision process (406) of the compositetranscription engine (404) is presented where, first, the particularaudio file is scored on a single pre-transcription audio characteristicof audio quality. For example, the audio quality score is determined tobe less than the value of “2” on a scale where “0.1” is the lowest valuedescribing a low audio quality and “5” is the highest value describingthe highest audio quality. In some embodiments, the audio quality scoremay strongly correlate with WER for a given audio file, where a negativecorrelation of 0.97 exists between the audio quality and WER for a givenaudio file reproduced with varying audio quality. For example, thecomposite transcription engine (404) is configured to compare the valueof the audio quality of the audio file to the specifications of themodels A (402) and B (403) to determine which model should be used forthe first pass of transcription. For example, because the audio qualityis less than “2”, the composite transcription engine (404) would choosethe model B as it was previously determined to be the best for audioquality of “2” or lower during the training phase. In this example, only“q” is evaluated for the “Q” metric above, and “1” and “a” are assumedto evaluate to “1”.

In some embodiments, the composite transcription engine (404) isconfigured to continue with the decision process after the model Breturns at least one hypothesis of the transcript, such as “testing onetwo three” or any series of words in text. In such case, for example,the model B outputs a confidence score of 0.6 for the model Bhypothesis. In some embodiments, the composite transcription engine(404) is configured to compare the outputted confidence matric value tothe confidence threshold (CT) in the model specification for model B,which is listed as 0.7. Because the confidence is lower than the model Bconfidence threshold in the specification, the model hypothesis would berejected for the moment. In some embodiments, the compositetranscription engine (404) is configured to then search for analternative model, which in this case (FIG. 4 ) model A is the onlyremaining model to conduct a second pass of transcription.

Again, in some embodiments, the composite transcription engine (404) isconfigured to continue with the decision process after the model Areturns at least one Model A hypothesis of the transcript and anaccompanied confidence score of 0.85. In some embodiments, the compositetranscription engine (404) is configured to compare the outputtedconfidence matric value to the confidence threshold (CT) in the modelspecification for model A, which is listed as 0.8. Because theconfidence is higher than the model A confidence threshold, thecomposite transcription engine (404) would accept the model Ahypothesis. In case, the outputted confidence matric value confidencewould have been lower than the model A confidence threshold, thehypothesis the composite transcription engine (404) would have rejectedthe model A hypothesis, and the previous hypothesis from model B wouldhave been accepted as the final hypothesis.

FIG. 5A shows a WER graph for four different models trained on variouscontent types (post-transcription property) and the audio quality(pre-transcription property) as tabulated in a table shown in FIG. 5B.As the graph of FIG. 5A illustrates the WER dramatically decreases whenall four speech recognition models are used, for example, in thetranscription of the new audio recording (405).

The word error rate of the complete test audio file transcriptioncomprising all segments by each of the models is shown in 501, where alower word error rate is more desirable and indicative of betterperformance. A model specification of all four models A, B, C, and Dutilized by the composite transcription engine outperforms allindividual models as well as smaller subset combinations of less thanfour of the four models. In this case, the composite 4 model engine hasa word error rate of 15.2%, compared to 23.2%-52% for the individualmodels. It should be noted that the 15.2% WER of the 4-model compositeruns through both phases of the analysis during decoding. If only phaseone is run and only the Q metric is evaluated to determine the bestengine, the WER increases to 19.2%.

During a transcription phase, the model specification is read by acomposite transcription engine containing all models and used todetermine which audio segments should be transcribed by which engine,and which hypothesis from the engines should be accepted or rejected. Anexample composite transcription engine is presented where the individualfour models achieve word error rate of 23.2% to 52% individually, but15.2% as a composite.

In some embodiments, the present invention provides for an exemplaryinventive computer-implemented method for improving computer speed andaccuracy of automatic speech transcription that includes at least thesteps of: generating, by at least one processor, at least one speechrecognition model specification for a plurality of distinctspeech-to-text transcription engines; where each distinct speech-to-texttranscription engine corresponds to a respective distinct speechrecognition model; where, for each distinct speech-to-text transcriptionengine, the at least one speech recognition model specification at leastidentifies: i) a respective value for at least one pre-transcriptionevaluation parameter, and ii) a respective value for at least onepost-transcription evaluation parameter; receiving, by the at least oneprocessor, at least one audio recording representing at least one speechof at least one person; segmenting, by the at least one processor, theat least one audio recording into a plurality of audio segments; wherein each audio segment corresponds to a respective single phrase of arespective single person that has been bounded by points of silence inthe at least one audio recording; determining, by the at least oneprocessor, based on the respective value of the at least onepre-transcription evaluation parameter of the respective distinct speechrecognition model in the at least one speech recognition modelspecification, a respective distinct speech-to-text transcription enginefrom the plurality of distinct speech-to-text transcription engines tobe utilized to transcribe a respective audio segment of the plurality ofaudio segments; submitting, by the at least one processor, therespective audio segment to the respective distinct speech-to-texttranscription engine; receiving, by the at least one processor, from therespective distinct speech-to-text transcription engine, at least onehypothesis for the respective audio segment; accepting, by the at leastone processor, the at least one hypothesis for the respective audiosegment based on the respective value of the at least onepost-transcription evaluation parameter of the respective distinctspeech recognition model in the at least one speech recognition modelspecification to obtain a respective accepted hypothesis for therespective audio segment of the plurality of audio segments of the atleast one audio recording; where the accepting of the at least onehypothesis for each respective audio segment as the respective acceptedhypothesis for the respective audio segment removes a need to submit therespective audio segment to another distinct speech-to-texttranscription engine from the plurality of distinct speech-to-texttranscription engines resulting in the improved computer speed and theaccuracy of automatic speech transcription; generating, by the at leastone processor, at least one transcript of the at least one audiorecording from respective accepted hypotheses for the plurality of audiosegments; and outputting, by the at least one processor, the at leastone transcript of the at least one audio recording.

In some embodiments, the generating of the at least one speechrecognition model specification for the plurality of distinctspeech-to-text transcription engines includes: receiving, by the atleast one processor, at least one training audio recording and at leastone truth transcript of the at least one training audio recording:segmenting, by the at least one processor, the at least one trainingaudio recording into a plurality of training audio segments and the atleast one truth transcript into a plurality of corresponding truthtraining segment transcripts; applying, by the at least one processor,at least one pre-transcription audio classifier to each training audiosegment of the plurality of training audio segments to generate firstmetadata classifying each training audio segment based at least on: i)language, ii) audio quality, and iii) accent; applying, by the at leastone processor, at least one text classifier to each corresponding truthtraining segment transcript of the plurality of corresponding truthtraining segment transcripts to generate second metadata classifyingeach corresponding truth training segment transcript based at least onat least one content category; combining, by the at least one processor,the plurality of training audio segments, the plurality of correspondingtruth training segment transcripts, the first metadata, and the secondmetadata to form at least one benchmark set; testing, by the at leastone processor, each distinct speech-to-text transcription engine of theplurality of distinct speech-to-text transcription engines based on theat least one benchmark set to form a plurality of model result sets;where each model result set corresponds to the respective distinctspeech-to-text transcription engine; where each model result setincludes: i) the at least one benchmark set, ii) at least onemodel-specific training hypothesis for each training audio segment, iii)at least one confidence value associated with the at least onemodel-specific training hypothesis, and iv) at least one word error rate(WER) associated with the at least one model-specific traininghypothesis; determining, by the at least one processor, a respective setof transcription decisions for each distinct speech-to-texttranscription engine of the plurality of distinct speech-to-texttranscription engines, where the respective set of transcriptiondecisions defines, for each distinct speech-to-text transcriptionengine, the value of the at least one pre-transcription evaluationparameter and the value of the at least one post-transcriptionevaluation parameter; and combining, by the at least one processor, eachrespective set of transcription decisions for each distinctspeech-to-text transcription engine of the plurality of distinctspeech-to-text transcription engines into the at least one speechrecognition model specification for the plurality of distinctspeech-to-text transcription engines.

In some embodiments, the at least one pre-transcription evaluationparameter is selected from the group of: i) the language, ii) the audioquality, and iii) the accent.

In some embodiments, the respective set of transcription decisionsincludes at least one of: i) a pre-transcription importance ranking of aplurality of pre-transcription evaluation parameters, and ii) apost-transcription importance ranking of a plurality ofpost-transcription evaluation parameters.

In some embodiments, the at least one post-transcription evaluationparameter is a confidence threshold.

In some embodiments, each segment of the plurality of audio segmentslasts between 5 and 15 seconds.

In some embodiments, the at least one audio recording is real-timestreamed audio of the at least one speech of the at least one person.

In some embodiments, the at least one audio recording includes at leasttwo speeches of at least two people; where the generating the at leastone transcript of the at least one audio recording further includes:generating a first transcript of a first speech of a first person, andgenerating a second transcript of a second speech of a second person;and where the outputting the at least one transcript of the at least oneaudio recording further includes: outputting the first transcript of thefirst speech of the first person, and outputting the second transcriptof the second speech of the second person.

In some embodiments, the at least two speeches are in distinctlanguages.

In some embodiments, the respective distinct speech recognition model isselected from the group consisting of: i) a phoneme-based acousticGaussian mixture model, ii) a phoneme-based acoustic hidden Markovmodel, iii) a phoneme-based acoustic neural net model trained fromforced phonetic alignments, iv) a phoneme-based acoustic neural netmodel trained without forced phonetic alignments, v) a character-basedacoustic neural net model, vi) any of i-v models coupled with an n-gramlanguage model, and vii) any of i-v models coupled with a generative,neural net language model.

In some embodiments, the present invention provides for an exemplaryinventive system for improving computer speed and accuracy of automaticspeech transcription that includes at least components of: at least onespecialized computer, including: a non-transient computer memory,storing particular computer executable program code; and at least onecomputer processor which, when executing the particular program code, isconfigured to perform at least the following operations: generating atleast one speech recognition model specification for a plurality ofdistinct speech-to-text transcription engines; where each distinctspeech-to-text transcription engine corresponds to a respective distinctspeech recognition model; where, for each distinct speech-to-texttranscription engine, the at least one speech recognition modelspecification at least identifies: i) a respective value for at leastone pre-transcription evaluation parameter, and ii) a respective valuefor at least one post-transcription evaluation parameter; receiving atleast one audio recording representing at least one speech of at leastone person; segmenting the at least one audio recording into a pluralityof audio segments; where in each audio segment corresponds to arespective single phrase of a respective single person that has beenbounded by points of silence in the at least one audio recording;determining, based on the respective value of the at least onepre-transcription evaluation parameter of the respective distinct speechrecognition model in the at least one speech recognition modelspecification, a respective distinct speech-to-text transcription enginefrom the plurality of distinct speech-to-text transcription engines tobe utilized to transcribe a respective audio segment of the plurality ofaudio segments; submitting the respective audio segment to therespective distinct speech-to-text transcription engine; receiving, fromthe respective distinct speech-to-text transcription engine, at leastone hypothesis for the respective audio segment: accepting the at leastone hypothesis for the respective audio segment based on the respectivevalue of the at least one post-transcription evaluation parameter of therespective distinct speech recognition model in the at least one speechrecognition model specification to obtain a respective acceptedhypothesis for the respective audio segment of the plurality of audiosegments of the at least one audio recording; where the accepting of theat least one hypothesis for each respective audio segment as therespective accepted hypothesis for the respective audio segment removesa need to submit the respective audio segment to another distinctspeech-to-text transcription engine from the plurality of distinctspeech-to-text transcription engines resulting in the improved computerspeed and the accuracy of automatic speech transcription; generating atleast one transcript of the at least one audio recording from respectiveaccepted hypotheses for the plurality of audio segments; and outputtingthe at least one transcript of the at least one audio recording.

While a number of embodiments of the present invention have beendescribed, it is understood that these embodiments are illustrativeonly, and not restrictive, and that many modifications may becomeapparent to those of ordinary skill in the art, including that theinventive methodologies, the inventive systems, and the inventivedevices described herein can be utilized in any combination with eachother. Further still, the various steps may be carried out in anydesired order (and any desired steps may be added and/or any desiredsteps may be eliminated).

What is claimed is:
 1. A computer-implemented method for improvingcomputer speed and accuracy of automatic speech transcription,comprising: generating, by at least one processor, at least one speechrecognition model specification for a plurality of distinctspeech-to-text transcription engines; wherein each distinctspeech-to-text transcription engine corresponds to a respective distinctspeech recognition model; wherein, for each distinct speech-to-texttranscription engine, the at least one speech recognition modelspecification at least identifies: i) a respective value for at leastone pre-transcription evaluation parameter, and ii) a respective valuefor at least one post-transcription evaluation parameter; wherein thegenerating the at least one speech recognition model specificationcomprises: receiving, by the at least one processor, at least onetraining audio recording and at least one truth transcript of the atleast one training audio recording; segmenting, by the at least oneprocessor, the at least one training audio recording into a plurality oftraining audio segments and the at least one truth transcript into aplurality of corresponding truth training segment transcripts; applying,by the at least one processor, at least one pre-transcription audioclassifier to each training audio segment of the plurality of trainingaudio segments to generate first metadata classifying each trainingaudio segment based at least on: i) language, ii) audio quality, andiii) accent; applying, by the at least one processor, at least one textclassifier to each corresponding truth training segment transcript ofthe plurality of corresponding truth training segment transcripts togenerate second metadata classifying each corresponding truth trainingsegment transcript based at least on at least one content category;combining, by the at least one processor, the plurality of trainingaudio segments, the plurality of corresponding truth training segmenttranscripts, the first metadata, and the second metadata to form atleast one benchmark set.
 2. The computer-implemented method of claim 1,wherein the at least one pre-transcription evaluation parameter is atleast one of: i) language, ii) the audio quality, or iii) the accent. 3.The computer-implemented method of claim 2, wherein the respective setof transcription decisions comprises at least one of: i) apre-transcription importance ranking of a plurality of pre-transcriptionevaluation parameters, or ii) a post-transcription importance ranking ofa plurality of post-transcription evaluation parameters.
 4. Thecomputer-implemented method of claim 1, wherein the at least onepost-transcription evaluation parameter is a confidence threshold. 5.The computer-implemented method of claim 1, wherein each segment of theplurality of audio segments lasts between 5 and 15 seconds.
 6. Thecomputer-implemented method of claim 1, wherein the at least one audiorecording is real-time streamed audio of the at least one speech of theat least one person.
 7. The computer-implemented method of claim 1,wherein the at least one audio recording comprises at least two speechesof at least two people; wherein the generating the at least onetranscript of the at least one audio recording further comprises:generating a first transcript of a first speech of a first person, andgenerating a second transcript of a second speech of a second person;and wherein the outputting the at least one transcript of the at leastone audio recording further comprises: outputting the first transcriptof the first speech of the first person, and outputting the secondtranscript of the second speech of the second person.
 8. Thecomputer-implemented method of claim 7, wherein the at least twospeeches are in distinct languages.
 9. The computer-implemented methodof claim 1, wherein the respective distinct speech recognition model isat least one of: i) a phoneme-based acoustic Gaussian mixture model, ii)a phoneme-based acoustic hidden Markov model, iii) a phoneme-basedacoustic neural net model trained from forced phonetic alignments, iv) aphoneme-based acoustic neural net model trained without forced phoneticalignments, v) a character-based acoustic neural net model, vi) any ofi-v models coupled with an n-gram language model, or vii) any of i-vmodels coupled with a generative, neural net language model.
 10. Thecomputer-implemented method of claim 1 further comprising: testing, bythe at least one processor, each distinct speech-to-text transcriptionengine of the plurality of distinct speech-to-text transcription enginesbased on the at least one benchmark set to form a plurality of modelresult sets.
 11. The computer-implemented method of claim 10 whereineach model result set corresponds to a respective distinctspeech-to-text transcription engine.
 12. The computer-implemented methodof claim 11 wherein each model result set comprises: i) the at least onebenchmark set, ii) at least one model-specific training hypothesis foreach training audio segment, iii) at least one confidence valueassociated with the at least one model-specific training hypothesis, andiv) at least one word error rate (WER) associated with the at least onemodel-specific training hypothesis.
 13. The computer-implemented methodof claim 12 further comprising: determining, by the at least oneprocessor, a respective set of transcription decisions for each distinctspeech-to-text transcription engine of the plurality of distinctspeech-to-text transcription engines, wherein the respective set oftranscription decisions defines, for each distinct speech-to-texttranscription engine, the value of the at least one pre-transcriptionevaluation parameter and the value of the at least onepost-transcription evaluation parameter.
 14. The computer-implementedmethod of claim 13 further comprising: combining, by the at least oneprocessor, each respective set of transcription decisions for eachdistinct speech-to-text transcription engine of the plurality ofdistinct speech-to-text transcription engines into the at least onespeech recognition model specification for the plurality of distinctspeech-to-text transcription engines.
 15. The computer-implementedmethod of claim 10 wherein each model result set comprises: i) the atleast one benchmark set, ii) at least one model-specific traininghypothesis for each training audio segment, iii) at least one confidencevalue associated with the at least one model-specific traininghypothesis, and iv) at least one word error rate (WER) associated withthe at least one model-specific training hypothesis.
 16. Thecomputer-implemented method of claim 15 further comprising: determining,by the at least one processor, a respective set of transcriptiondecisions for each distinct speech-to-text transcription engine of theplurality of distinct speech-to-text transcription engines, wherein therespective set of transcription decisions defines, for each distinctspeech-to-text transcription engine, the value of the at least onepre-transcription evaluation parameter and the value of the at least onepost-transcription evaluation parameter.
 17. The computer-implementedmethod of claim 16 further comprising: combining, by the at least oneprocessor, each respective set of transcription decisions for eachdistinct speech-to-text transcription engine of the plurality ofdistinct speech-to-text transcription engines into the at least onespeech recognition model specification for the plurality of distinctspeech-to-text transcription engines.